Skip to content

Conversation

DWesl
Copy link
Collaborator

@DWesl DWesl commented Sep 25, 2019

Description

Try to provide determinants for the various linear operators.

Motivation and Context

I would like to optimize the correlation length parameters as well as the mean fluxes. This requires the full likelihood, in particular, the determinant of the covariance matrix. I can't calculate that at the moment. This takes a few steps toward calculating those

How Has This Been Tested?

I check the determinants of some operators against numpy.linalg.det(op.dot(np.eye(*op.shape))).
The Fourier operators are causing problems.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I give permission for my code to be distributed under the existing license

…se).

It was dropping the first and last elements along the first axis every
time, instead of along the axis being concatenated.
The primary data stored by this class is a list of eigenvalues, so
this should be easy.  For some reason it's broken for all but trivial
cases.  I can get the determinant to be one for diagonal correlations,
but this is not the primary class for diagonal operators, especially
not ones proportional to the identity matrix.  I have no idea why this
doesn't work.  This may be enough for the purposes of this package,
since it's usually only off by 50% or so, but I really don't like
calling the result a `determinant`.
@DWesl DWesl added enhancement New feature or request help wanted Extra attention is needed labels Sep 25, 2019
DWesl added 8 commits October 17, 2019 18:34
Non-cyclic homogeneous correlations are still broken.
I still can't figure out how to get those working.
…se).

It was dropping the first and last elements along the first axis every
time, instead of along the axis being concatenated.
The primary data stored by this class is a list of eigenvalues, so
this should be easy.  For some reason it's broken for all but trivial
cases.  I can get the determinant to be one for diagonal correlations,
but this is not the primary class for diagonal operators, especially
not ones proportional to the identity matrix.  I have no idea why this
doesn't work.  This may be enough for the purposes of this package,
since it's usually only off by 50% or so, but I really don't like
calling the result a `determinant`.
Non-cyclic homogeneous correlations are still broken.
I still can't figure out how to get those working.
Gets tox to try it again.
…n apreriodic tests.

At this point just under half of the aperiodic tests are broken.  Some
of them are close and would pass if I required only three-decimal
precision, others differ by an order of magnitude.
@DWesl DWesl force-pushed the operator_determinants branch from da13bd8 to e5f5fb5 Compare November 8, 2019 18:59
@DWesl
Copy link
Collaborator Author

DWesl commented May 28, 2020

This works with the spectral operators on periodic domains. I cannot get this to work for spectral operators on non-periodic domains. There does not appear to be any literature on the subject of determinants of Toeplitz and block Toeplitz operators, although there are suggestions that the determinants can be calculated by means of Levinson-Durbin recursion, somewhat as a side product of solving the linear system.

@DWesl
Copy link
Collaborator Author

DWesl commented Aug 3, 2020

An additional complication: many applications will have different correlation lengths for land and ocean, or will optimize only the one or the other. In either case, the easy way to do that sets many standard deviations to zero, which forces the determinant to zero. Making this useful requires embedding and restriction operators to embed either the land or ocean points within the rectangular domain, apply the transforms, then pull out only the land or ocean points to pass to the next section of the program.

If optimizing only net fluxes for land and ocean, a separate operator to rearrange fluxes from a grid so that land fluxes are next to each other, as are ocean fluxes. If optimizing GPP and Reco separately, or fossil fuel and biogenic fluxes separately, being able to rearrange a net flux map to separate the ocean and land fluxes is less useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request help wanted Extra attention is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant